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This work addresses the following question. Let G be a (real or complex) linear algebraic reductive group 
acting on an affine variety V and let be a subvariety of interest. 

Question. Can we calculate the size of the moduli of G- orbits intersecting W? 

By moduli we mean the set of points up to the equivalence of lying in the same G orbit. More generally, 
we are interested in understanding W up to G-equivalence; here G docs not preserve W. We study this 
question when W is smooth and there exists a reductive subgroup H oi G which measures the G-action 
along W. The notion of measuring the G-action is by means of the moment map and we say that G is 
i/-detectcd along W if mc{w) G i) — Lie H for w £ W , where mc is the moment map of the G action (see 
Section [T]). Our main result is the following. 

Theorem 12.11 Let G, H, V, W be as above. Suppose G is H -detected along W. For w ^ W <Z V , the 

components of G ■ w C\W are Hg-orbits, where Hq is the identity component of H . Consequently, G ■ wClW 
is a finite union of H- orbits. 

As an application, we apply our work to the problem of finding continuous families of non-isomorphic 
nilpotent Lie groups which do not admit left-invariant Ricci soliton metrics (Section [3]). Additionally, this 
work can be applied to finding continuous families of non-isomorphic nilpotent Lie groups which do admit 
left-invariant Ricci soliton metrics, see |Jab08cj . Another application of the above theorem to nilgeometry 
is the foUwing. 

Theorem 13.51 Let N be a nilpotent Lie group such that N = N1N2, a product of normal subgroups. Then 
N admits a left-invariant Ricci soliton metric if and only if both Ni and N2 admit such a metric. 

We finish by demonstrating our techniques applied to the adjoint representation (Section |4|). Let G be a 
semi-simple, or reductive, group and consider the adjoint action on g. We reprove the classical result that 
there are finitely many nilpotent orbits and show that every nilpotent orbit in g is (analytically) distinguished 
in the sense of Definition 11.61 that is, each nilpotent orbit contains a critical point of the norm squared of 
the moment map. 

1 Preliminaries 

Let G be a complex linear reductive group acting (rationally) on an affine algebraic variety X. The following 
theorem is well-known. 

Theorem 1.1. There exists a linear representation T : G ^ GL(V) and closed imbedding i : X V which 
is G-equivariant; that is, i{gx) — T{g)i(x) for x e X , g G G. 

In this way we can reduce to the setting X ^ V, for a proof see [PV94| Theorem 1.5]. Let G be a complex 
reductive group and K a maximal compact subgroup of G. Let y be a complex vector space on which G 
acts linearly and rationally. We denote this action by 

GxV^V {g,v)t-^g-v 
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We may endow V with a positive definite Hermitian inner product H. This Hermitian inner product may 
be written as H — S + iA where S, A are real-valued symmetric, (resp.) anti-symmetric real bilinear forms. 
Here S = ^{H + H}, A ~ -^{H — H}, and A{v,w) — S{v,iw), where z denotes the complex conjugate of 
z e C. Equivalently, we could start with an inner product (real bilinear, positive definite form) S which is 
i-invariant and build a positive definite Hermitian form H as above. 

A Hermitian inner product H(-, ■) on V is called G-compatible if H is A'-invariant. This definition implies 
that G is closed under the adjoint operation. Like wise, a (real) inner product S is called G-compatible if 
it is /iT-invariant and it acts symmetrically, where g = t (B it and t = LK; that is, if it is if- invariant and 
i-invariant. Such inner products always exist on V by averaging. We will denote S{-, ■) by < ■, ■ > also. 

Cartan Involutions 

Let G be a closed subgroup of GL{V). The adjoint with respect to H is denoted by *. Any involution of 
the form 9{g) = {g*)~'^ for g e GL{V) is called a Cartan involution of GL{V). If the involution 6 leaves G 
stable, then we say that 6* is a Cartan involution of G. Such involutions always exist as we demonstrate. 

Choose H, or equivalently S, to be invariant under K and i, as above. Then G is stable under the 
metric adjoint operation. This gives a Cartan involution 9{g) ~ {g*)^^ on G with K — G^ . We denote the 
corresponding involution on g by 61 and we have t = LK = +1 eigenspace of 9 and p = it — —1 eigenspace 
of 9. We observe that on g is conjugate linear and is just complex conjugation with respect to the real 
form t of 0. See jMos55j . [RS90| . and references therein for more information on Cartan involutions and 
decompositions. 

Proposition 1.2 (Mostow). Let G he a complex (linear algebraic) reductive group and H a reductive sub- 
group. Then there exists a Cartan involution 9 which simultaneously preserves G and H . The involution 9 
can be chosen so that Kh — is a previously chosen maximal compact subgroup of H and Kh C K — G^ 
where K is a maximal compact subgroup of G. 

We say that the Cartan decomposition of H above is compatible with the Cartan decomposition of G. In 
practice we will be interested in inner products which are both G and i/-compatible. The above proposition 
says that such inner products always exist. If an inner product is G-compatible and H is compatible with 
G, then the given inner product is i/-compatible. 

Similarly Cartan involutions exist on real algebraic reductive groups and the above proposition is still 
valid. In the real setting, however, the space p = —1 eigenspace of 9 will not equal it as g might not be 
a complex Lie algebra. Regardless, Cartan involutions and the decomposition g = t © p are good tools 
for studying the the geometry of orbits of real rational representations of real reductive groups, see |RS90| . 
[MarOlj . and [EJ08 for apphcations to the study of real group orbits. 

Moment maps 

Endow Q with an Ad if -invariant, z-invariant, ^-invariant inner product <<,>>. This is equivalent to 
<<, >> being the real part of an Ad if -invariant, ^-invariant Hermitian inner product; we choose <<, >> 
to be 6'-invariant so that g = t © p is an orthogonal decomposition. Such inner products on g always exist as 
demonstrated by the following examples. Additionally, one could guarantee their existence by averaging. 

Example 1.3. Let G he an algebraic reductive subgroup of SL{E). Let 9 denote a G stable Cartan involution 
of SL{E) and let B denote the Killing form of 5[{E). Then the inner product << •, • >>= —B{-, 9{-)) satisfies 
the conditions stated above. 

Example 1.4. If G is semi-simple then one may use the inner product << ■, ■ >>= —Bg{-,9{-)) where Bg 
is the Killing form of Q. 

Consider the identification/isomorphism ip : g ^ g* = HomT^{g,M.) defined via <<,>>; that is, 
(p{X) =<<X,- >>. If we consider the coadjoint action of G on 0* defined by (fc-i^)(y) ^ F{k^^ -Y) then the 
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isomorphism Lp is /sT-equivariant. We endow g* with the inner product << Lp{X)^ip{Y^ >>=<< X^Y >> 
so that if is an isometry. This inner product on g* is iiT- invariant (with respect to the coadjoint action). 
Moreover, we have 6* ~ J and p* — p and the decomposition g* = 6* © p* is orthogonal. 

We define the g*-valued moment map m* : V Q* /t* ~ p* C g* as follows. First, consider the function 
Pt, : G ^ M defined by Pv{g) = \g ■ wp, where =< u, u >— S{u, u) = H{u, u). The function m* : ^ g* 
is defined by 

m*{v) = d{p^)e 

The ivT-invariance of | ■ p implies m*{v)\t = 0. 

We define the g- valued moment map to : — > g by to* = Lp{m). Equivalently, to is defined implicitly by 

«m{v),X »=2 < X - v^v > for aU X e g 

The ii'-invariance of <, > implies m{v) G C g. The 0-invariance of <<, >> implies = p and thence 

m{v) G p for all u G F 

Having p-valued moment maps is the reason we required <<, >> to be invariant. 



We observe that m, to* are (real) homogeneous polynomials of degree 2. Moreover, for c G C we have 
m(cv) = |cpTO(v) and m*{cv) = \c\'^m* [v), where |c|^ = c c is the usual norm square on C. This gives rise 
to well-defined polynomials on complex projective space 

r 1 rn{v) , m*{v) 

The norm of these functions is the same. Historically more attention has been placed on to*, however to 
study ||to|P — ||m*|p we will work with the function to as it is very natural from the perspective of the 
problems addressed in this note. 

When G is a real reductive group the same results hold except that p is not necessarily it. Moreover, in 
the real setting our moment maps give rise to well-defined functions on real projective space. 



Closed and distinguished orbits 

Consider the action of G on V . It has been shown that the closed orbits are precisely the orbits which 
intersect the zero set of the moment map m : — > p. In the complex setting this theorem was done by 
|KN78| and in the real setting done by |RS90j . We state it below. 

Theorem 1.5. Consider the action G x V V . Denote the zero set o/ to : — > p by dJt = m~^{Q). For 
V d V , the orbit G ■ v is closed if and only if G ■ v CldJl ^ 

The set of points whose G-orbits are closed is called the set of stable points. In contrast, a point is called 
unstable if its G-orbit contains zero. The set of unstable points is called the null-cone. This set of points 
can be studied in a more refined way by passing to projective space. 

Definition 1.6. A point v £ V or [v] ¥V is called G-distinguished, or just distinguished when the G 
action is clear, if [v\ G PV^ is a critical point of ||to|P : TV M. Likewise, we call an orbit G ■ v, or G ■ [v], 
distinguished if it contains a distinguished point. 

Closed orbits are always distinguished as zero is an absolute minimum of ||to|P. The non-closed distin- 
guished orbits all lie in the null-cone. 

Lemma 1.7. Let v G V . Then v is distinguished if and only if m{v) ■ v — cv for some c G C. 
We omit the proof of this lemma as it follows immediately from the definitions. 
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Theorem 1.8. Let G, q, K , and V he as above endowed with inner products as above. Let m denote the 
moment map of the representation and, for v G V, denote by ft{v) the negative gradient flow of 
starting at v. Denote the limit point of this flow by ipodv). If G ■ v is a distinguished orbit with v such a 
distinguished point, then for every g £ G, fooig ■ v) E K ■ v. 

This theorem is true in both the real and complex settings, see |Jab08a| . A priori, it is not clear that 
the limit set of the flow is a single point. For this point and more information on moment maps, see [Sja98[ 
Section 2.5]. 

Detecting G-orbits along subvarieties 

Let G be a (real or complex algebraic) reductive group which acts linearly and rationally on V. Let H 
be a reductive subgroup of G which has a compatible Cartan decomposition (see Proposition 1 1.2p . Let V 
be endowed with a G-compatible metric (see beginning of Section [ij. Recall that the G-compatible metric 
on V is also i7-compatible as H has a compatible Cartan decomposition. Let W be an _ff-stable smooth 
subvariety of V. 

Definition 1.9. We say that the G-action on V is 'LI -detectable along W ' if mc{w) S f) for w € W . 
Similarly, we could state this definition for smooth varieties in projective space. 

We observe the following. Let y be a G-representation which is if-detectable on W . Then mciw) = mniw) 
for all w S W. The proof follows immediately upon writing out the definitions, as we demonstrate. As 
mciw) € f), we can show the desired equality by comparing the inner products of mciw) and mniw) with 
every element oi X E t) 



2 Orbits of Compatible Subgroups 

Let G be a reductive group with a fixed choice of Cartan decomposition relative to a Cartan involution 9. 
Let H he a reductive subgroup which is compatible with the choice of Cartan decomposition; that is, H is 
6'-invariant (cf. Proposition II. 2p . If G acts on V and is endowed with a G-compatible inner product <, >, 
then the inner product <, > is iJ-compatible; that is, LI is self-adjoint. 

Let tuq, resp. mn, denote the moment map of G, resp. LL, acting on V. Suppose there exists an _ff-stable 
smooth subvariety W C V on which the G-action is _ff-detectable; that is, such that mc{w) — mniw) for 
dWwEW (see the definition above). The subvariety W is not required to be closed. 

The following theorems are true for real and complex groups. We first give proofs for complex groups 
and finish the section by explaining how to extend the results over C to results over R. 

Theorem 2.1. Let G, LI, W, V be as above. If w £ W C V, then the components of G -wHW are H^-orbits 
and, consequently, G ■ w C\W is a finite union of H -orbits. 

Remark. In this way we obtain a solution of the original question. The dimension of the moduli of G 
orbits which intersect W is precisely the dimension of the moduli of H orbits in W . Moreover, if the H 
action on W were stable, then a lower bound on the dimension of the moduli of H orbits would be the 
dimension of W//H, the GIT quotient. See |MFK94j for more information on Geometric Invariant Theory 
(GIT) and quotients. 

Corollary 2.2. Let G, H, W, V be as in the theorem above. Then for w € W , the intersection G ■ w DV is 
smooth. 



< mciw), X > 







< X ■ w,w >v 

< m,H{w),X >!, 

< mniw), X >g 



from the definition of the moment map 
from the definition of the moment map 
as the metric on () is the induced metric from g 
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Theorem 2.3. Let G, H, W, V be as above but with W a cone in V ; that is, W descends to a projective variety 
fW ofW. Consider the induced actions on Py,PVF. // [w] G VW , then the components of G ■ [w] OWW are 
Ho-orbits and, consequently, G- [w] nPW is a finite union of H -orbits. 

Remark. The proof of the second theorem does not foUow immediately from the first. However, the 
proofs are very similar and we give them simultaneously. 

Corollary 2.4. Let w W d V where W is a closed H-stable smooth subvariety (e.g., a subspace). Then 
G ■ w is distinguished if and only if H ■ w is distinguished. 

Remarks. (1) Closedness of W is necessary in this corollary as finding distinguished points involves taking 
limits. This corollary gives a useful criterion for determining when a specific orbit of G or iJ is distinguished. 
We give several worthwhile applications of this corollary in the following section and also in [Jab08c| . 

(2) One of our main applications is to study the negative gradient flow of the norm squared of the moment 
map. A problem of interest is to understand when the limit points of this flow are contained in the orbit of 
the initial point. However, Theorems 12.11 and 1 2 . 31 can be applied to much more general settings of evolutions. 
If one knows that one can evolve within a special subgroup and that one has convergence in the large group 
orbit, then one can achieve convergence within the subgroup orbit. This is a 'specialized change of basis' 
type result, cf. Section [H 

(3) It is not true, in general, that H ■ w must be distinguished ii G ■ w is distinguished. In the special 
case of closed orbits, this problem has been explored in [VinOO] and [EJ08| . In [VinOOj it is shown that if 
G • ui is closed then H ■ gw is closed for generic g ^ G. Other criteria to determine closedness of orbits of 
(sub)groups have been constructed in |EJ08j : at the moment there is no general criterion which completely 
determines when H ■ w is closed, even ii G ■ w is closed. For an explicit example of a non-closed H orbit in 
a closed G orbit, see |Jab08b| . 

Before proving the theorem, we use it to deduce Corollarv l2.4l 

Proof of the Corollary \2.4\ We apply Theorem 12.31 and Theorem 11.81 to prove the corollary. 

Recall that [w] G PW^, or w G W , is a G-distinguished point if (by definition) [w] is a critical point of 
I Tog IP '■ ^ R where tog is the moment map of the G-action on TV . We denote the moment map on V 
and f'V by the same notation as context should avoid any confusion. It is well-known, see Lemma 1 1.71 that 
[w] G PVF C is G-distinguished if and only if mQ{w) ■ w = cw for some c G C. In this way we see that a 
point w & W is //-distinguished if and only if w G is G-distinguished as mG{w) — mH{w). Thus, ii H -w 
is iJ-distinguished, then G • w must be G-distinguished. 

Conversely, consider w G W such that the orbit G ■ w is G-distinguished. Theorem 11.81 provides the 
existence of gn G Go and g G G such that [gn ■ w] — *■ [g ■ w] in ¥W, as n ^ cx), and [g ■ w] is distinguished. 
Notice that the sequence [gnw] can be chosen to lie in PW by following the negative gradient flow of ||tog||^ 
starting at [w] and our limit point is in FW as PW is closed. Moreover, observe that [g ■ w] is in the same 
connected component of G • [w] n PW as [w] . By Theorem 12.31 there exists h G H such that 5 • [w] — h ■ [w]; 
that is, H ■ [vu] contains a distinguished point and hence H ■ w is a distinguished orbit. 

□ 

Next we prove Theorems 12. II and 12.31 The proofs of both are given simultaneously as they are so similar. 
It suffices to consider H which is connected as H, being an algebraic group, has finitely many components. 

Lemma 2.5. Forw G W, Ty,{G ■w)riTy,W ^Tyj{H-w). When W is a cone in V with projection PW C PV, 
we have T[^](G • [w]) n T[^]PW = T[^]iH • [w]). 

Proof of the lemma. To show equality, we show containment in both directions. One direction is trivial. 
Since H preserves W, we immediately have Tw{H-w) C T^{G ■ w) HT^W which then imphes T[„](i/ ■ [w]) C 
T[w]{G ■ [w]) n T[^]PW. The reverse containments are shown below. 

We prove the lemma at the affine level first. By translation, we have the following identifications 

r^(G-u.) ~ 3-w 
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Recall that our Cartan decompositions g = t p and I) — tn ® Ph satisfy tn C t, \) — H, and = i^H- 
Take X a p Q pH , the orthogonal compliment oi pH in p, then 



=< X, mG{w) >=< X -w^w > for all It; e 

as mciw) = niH{w) G f). Consider G T.^jW and a curve 7(t) e W tangent to Vw s.it = 0; that is 7(0) = w 
and 7'(0) = v. Applying the above equation to this curve and differentiating we obtain 



= — 



<X-7(t),7(t) > 



i=0 



= < X ■ v,w > + < v,X ■ w > 
= 2 < X ■ w,v > 



as X = X* e pQpH C p is symmetric with respect to <, >. Thus, X -w 1. T^W for w G and X e p 0pH. 

Since T^^W is a complex vector space and <, > is i-invariant, for w e W , Vw € T^W, and X € p 0pif we 
have =< X-w, -~iv >=< iX-w, v >:= < (iX) ■w,v>asQ acts C-linearly. Since gef) = (6©P)0(6h©Ph) = 
C — span{p Ph} we have 



X -w ± T^W for w e T4^, X EgQt) 



(2.1) 



Now pick X £ g such that X ■ w e g ■ w n TyjW . Writing X = Xi + G [} (g ()), we obtain 
Xi- w + X2 ■ w G TujW which then implies X2 ■ w G TwW as H acts on by hypothesis. Here we are using 
the smoothness of W to insure that X2 ■ w = X ■ w — Xi ■ w £ T^W . But then X2 ■ w = by Equation 12. II 
Hence, X ■ w — Xi • w G f) • w as desired. This proves the lemma in the affine case. 



To prove the second part of the lemma, we reduce to the first part. Recall that the map n : V ^ ¥V is a 
submersion and the restrictions it : G-v ~» G- [v] and tt : W ^ VW are surjective. Let tt* : TV TPV denote 
the induced map on the tangent bundles and recall that if Vw € T^V is such that 7r*(w^) G T^^^FV, then 
t; G C < >. 

Consider G T[^]iG ■ [w]) n T[^]VW. Then there exist X^^ G T^(G • w) and X'^ G T^W such 

that = TT^iX'J = n^X'^) G r[^]PVF. This imphes tt^{X^ - X^) = e T^^fW, which implies 

X'^ — X'^ £ C < w >m, which then implies X^ G X^ + C < w >u)C T^W as W is a cone and smooth at 
w ^ 0. 

That is, XIj G T^(G • w) D T^W = Tyj{H ■ w) by the first part of the lemma and so = t:*{X'^) G 
Ti.{T^H ■ w) = T[„](if • [w]). Thus, T[„](G • [w]) n T^^fW = T^^^{H ■ [w]). 

□ 

As the lemma is now proven, we continue with the proof of the theorems. If we knew G ■ w nW were 
smooth, then by dimension arguments and the lemma above it would be easy to establish that G-wCiW ~ H-w 
locally near w. A priori, however, we cannot guarantee smoothness; obviously the theorems show smoothness 
a posteriori. We recall the following basic proposition on algebraic group actions; for a proof see [Bor9H 
1.1.8]. 

Proposition 2.6. Let G be an algebraic group acting on a variety X. Then for x G X, the boundary 
d{G ■ x) — G ■ X — G ■ X consists of G- orbits of strictly lesser dimension. Moreover, the orbit G ■ x is open 
and dense in G ■ x. The Hausdorff and Zariski closures of the orbit coincide. 

Lemma 2.7. Let G be an algebraic group and let H be a closed subgroup of G. Let G act on a variety V 
and assume that W is an H -stable subvariety. Suppose the following is true for every w (^W 

T,,{G-w)r\T^W = T^{H -w) 

Then the intersection of each G-orbit with W is a union of finitely many H-orbits. 
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Clearly this lemma combined with the previous lemma completes the proof of the theorems. We would 
like to thank Chuck Hague for pointing out to us that this lemma previously existed in the literature (see 
[Jan04[ Lemma 2.4]). 

Proof of the lemma. Let w G W, we can decompose the variety G ■ w HW = UXi into irreducible com- 
ponents Xi. As we are interested in the component oi G ■ w DW that contains w, we only need to consider 
the irreducible components Xi that contain w. 

We begin by showing H preserves Xi. Take pi £ Xi which is a smooth point of Xi. Then G ■ pi DW 
coincides with Xi near pi (in the Hausdorff sense), and so h ■ pi ^ Xi ior h Oe C H where Oe is a 
(Hausdorff ) open neighborhood of the identity element e E H . Let ^ : G xV ^ V denote the G-action on 
V . Since Oe — H and ^ is continuous we have 

KH,Pi) = fJ,{Oe,Pi) C fi{Oe,Pi) C X, 

This holds for any smooth point of Xi. Let 14 denote the set of smooth points of Xi. Then U = Xi (cf. 
[Sha94|, Lemma 2.1.1]) and as before 

fi{H,Xi) = ii{HM) C WiM) C U pi{H,p.,) C [j X^^X, 

This shows H acts on Xi. 

Now that H ■ Pi G Xi we may compare the following dimensions 

dim H ■ Pi < dim Xi at pi < dim(Tp; [G ■ pi) fl Tp^ W) — dim Tp. {H ■ pi) 

where the second inequality follows from Tp^ {G-piCi W) C Tp. (G-pi) PlTp . W and the last equality follows from 
the lemma above. Thus dhnH ■ pi — dimX^ and we see that dimH ■ pi is an (analytic) open neighborhood 
of Pi in Xi. As Xi is irreducible, we see that H ■ pi = Xi. Since H ■ pi is open and dense in H ■ pi, the same 
is true for H ■ pi C Xi. 



Next we show H ■ pi = Xi. Let w' € Xi. First we observe that w' G H ■ pi, as stated in the previous 
paragraph. The next lemma will use the fact that the Hausdorff and Zariski closure of the orbit coincide (cf. 
Proposition 12. 6|) . 

Sublemma 2.8. dim T^-(G • w') DT^'W > dimff • 

Proof of the sublemma. By hypothesis G ■ w' — G ■ pi and there exists Wn ^ H ■ pi such that Wn — > w' as w' 
is in the Hausdorff closure of H ■ pi. 

Pick an orthonormal basis {X"}^^]^ of T^^H ■ Wn — T^^H ■ pi where r — dimiJ • pi. By passing to a 
subsequence we may assume — > Xj as n — > cxd, for j = 1, . . . , r. Since G-w' = G-pi we have Xj G T^'G-w' 
which then implies {Xj}^^i C Tw'{G-w')nTw'W. As this collection of vectors is orthonormal, the sublemma 
is proven. 

□ 

By hypothesis and the above work, we have 

dimH ■ w' ^ din\Ty,,{H ■ w') = dim(T^/G • w' n T^'W) > dimH -p. 

But w' ^ H ■ Pi. Proposition 12 . 61 implies dimH ■ w' < dim_ff ■ pi with equality if and only if H ■ w' — H ■ pi. 
Applying this fact and the inequality above, we see that H ■ w' = H ■ pi. Since w' G Xi was arbitrary we 
have H ■ Pi = Xi. We observe that we have shown Xi — H ■ w' for any w' G Xi . 

Recall that we have decomposed G ■ w D W — UXi into irreducible components. If an irreducible com- 
ponent Xi contains w then Xi = H ■ w, as shown above. Thus the topological component oi G ■ w {^W 
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containing w is Xi — \^ H ■ w = H ■ w. This proves the lemma and the proofs of our theorems 

{Xi\ wex,} 

are complete. 

Remark. We point out that this technique of using compatible subgroups to study G ■ w D W does 
not completely detect the phenomenon of the intersection being a finite union of i?-orbits. There do exist 
examples where W is an 7J-stable subspace, each intersection G ■ w nW is finite union of iJ-orbits, but that 
the G-action is not iJ-detectable along W for any choice of inner products on V and g. This is proven by 
constructing H and W so that the G-orbit through a generic point of W is closed, but so that the iJ-orbit 
through said points is not closed (cf. Corollarv l2.4p . 

The question of closed orbits of subgroups 

Let G be a reductive group acting rationally on V. Let iJ be a reductive subgroup. If G • w is closed, then 
it is known that H ■ gv is closed for generic g E G. This problem has been worked on by many people, see, 
e.g., [Lun72j . [Nis73| . |VinOOj . or [Jab08b| . However, it is not true for all 5 G G that H ■ gv must be closed. 
It is an interesting problem to try and determine when the orbit of H is closed. 

Corollary 2.9. Suppose there exists an H -stable smooth (closed) subvariety W along which G is H -detected. 
If G ■ w is closed, then so is H ■ w. 

This is the special case of Corollary 12.41 when our distinguished orbit is closed. We ask the following 
question. 

Question 2.10. Let H be a reductive subgroup of G. Let G act rationally on V an suppose that G ■ v and 
H ■ V are closed. Do there exist inner products on V and q, satisfying the hypothesis of Theorem \2.1[ such 
that G is H -detected along the smooth subvariety H ■ v? 

To the contrary we could ask 

Question 2.11. Does there exist v E V such that G ■ v is closed, G is H-detected along H ■ v, but H ■ v is 
not closed? 

As stated at the end of the previous subsection, there do exist examples of V, W, G, H such that G cannot 
be i7-detected along W for any choice of inner products on V and g. 

Real algebraic groups 

Here we explain how to obtain the above results over M. Let G be a real algebraic reductive group. The real 
group G can be realized as the real points of a complex algebraic reductive group G"' such that G is Zariski 
dense. This is well-known and the construction of such a group G"' can be found in [JabOSaj . for example. 

The following result of Borel-Harish-Chandra (|BHC62j Proposition 2.3]) is the standard way of relating 
the real and complex settings. Let V be real vector space on which G acts linearly and rationally. Let 

— V ®C denote the complexification, then G"" acts linearly and rationally on V'^. 

Theorem 2.12. Consider v V C F"'. Then G"' ■ v CiV is a finite union of G-orbits. Moreover, the orbit 
G^ • v is closed if and only if G ■ v is closed. 

In the last assertion, the only if direction requires the work of either |Bir71| or [RS90| . In fact, a slightly 
stronger version of this theorem is true. We state this version below and refer the reader to [Jab08a| for a 
proof. 

Theorem 2.13. Consider v eV C V"" . Then G^ • v is distinguished if and only if G ■ v is distinguished. 

To obtain Theorems 12 . II and 12 .31 over M, one just needs know that they are true over C and apply Theorem 
12.121 To obtain Corollary [231 over M, one just needs to know that it is true over C and apply Theorem l2.13l 
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3 Applications to the Left-Invariant Geometry of Lie Groups 



The theorems in this section can be viewed as specialized change of basis theorems. We are interested in 
left-invariant Ricci soliton metrics on nilpotent Lie groups. Such a metric is called a nilsoliton and if a 
nilpotent Lie group admits such a metric, it is called an Einstein nilradical (see |LW07] for justification of 
this terminology). 

Question. Which nilpotent Lie groups are Einstein nilradicals? If a nilpotent Lie group admits such a 
metric, how can one find this special metric? 

As a left-invariant metric on a Lie group is equivalent to an inner product on its Lie algebra, we reduce 
to studying Lie algebras with inner products. The nilsoliton condition can be completely phrased at the 
algebra level. 

There are two points of view that one can take. The first is to fix a Lie bracket on a vector space and 
vary the inner product, the second is to fix an inner product on a vector space and vary the Lie bracket. 
We will work from the second perspective as it has produced many results and allows us to exploit tools 
from Geometric Invariant Theory. This is the perspective taken by J. Lauret and others (see [Lau06j and 
references therein). 

These special metrics are realized as critical points of the norm squared of the moment map corresponding 
to a particular G'i(n,R) action. Let 01 be an n-dimensional real vector space with fixed inner product <, >. 
Consider the space V — A^(*Jl)* (8) This is the space of skew-symmetric bilinear forms from 01 x to 01. 
The set of Lie brackets on 01 is a variety V in V. The change of basis action of GL{n,M.) on 01 induces the 
following action on V 

g■^i{X,Y)^g^^ig-'X,g-'Y) 

for g £ GL{n, M), fj. ^ V, and X,Y € M". This action preserves the variety V of Lie brackets. Moreover, if 
/X £ V is a Lie bracket on 01, then the orbit GLn ■ /i is precisely the isomorphism class of /i in V. The metric 
nilpotent Lie algebra with bracket /i and inner product < , > is denoted 01^ and the simply connected nilpo- 
tent Lie group with Lie algebra 01^ and left-invariant metric corresponding to <, > is denoted by {iV^, <, >} 
or just Nf_i when <, > is understood. Let g* < ■,■ >=< g~^-,g~^- > for g G Gi(n,M), then {A^g.^, <, >} is 
isometric to {Np_,g* <, >}. Thus iV^ admits a nilsoliton metric if and only if Ng.^ is a nilsoliton for some 
g € GL(n, M) . In this way we can study the set of left-invariant metrics on A^^ by studying the orbit g • /i in V. 

The inner product on 01 extends naturally to an inner product on V defined by < A, /i X^y /c 
X{Xi, Xj), Xk >< iJ.{Xi, Xj), Xk >, where {Xi} is any orthonormal basis of 01. This inner product on V is 
0(n)-invariant, where 0{n) is the orthogonal group relative to <, > on 01. 

A Cartan decomposition of gl{n) is given by 5o{n) © symm{n) where 5o{n) is the Lie algebra of 0{n), 
the skew-symmetric endomorphisms relative to <, >, and symm{n) is the set of symmetric endomorphisms 
relative to <, >. We consider the inner product << A,B >>= tr{AB*) on Ql{n), where the transpose is 
determined by <, >. This gives rise to the following moment map (see |Lau06[ Proposition 3.5]) 

mill) = -4J2iad^X^yad^X, + 2j2ad^Mad^X,Y (3.1) 

i i 

where {Xi} is any orthonormal basis of 01. Or equivalently, for X, F G 01, 

< m{ii)X,Y>= -4^ < ii{X,X,),X, >< fiiY,X,),X, > +2^ < fi{X,,X,),X>< iiiX,,Xj),Y > 

(3.2) 

Theorem 3.1 (Lauret). The Lie group (with left-invariant metric <,>) is a left-invariant Ricci soliton 
if and only if [/i] is a critical point of : PV^ ^ K; here [/i] denotes the class of /i in PV. Equivalently, 

is an Einstein nilradical if and only if the orbit GL{n,M.) ■ fj, is distinguished. 
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Given /i G V, we are interested in the problem of finding g G GL{n, M.) such that Ng.^ is a nilsoliton. The 
following theorems say that g can be chosen from a subgroup of G'L(n,]R) that reflects natural symmetries 
in the metric algebra 9^^;. In this way, the following are considered specialized change of basis theorems. 

Theorem 3.2. Let {N, <, >} be a nilpotent Lie group with left-invariant metric <, >. Denote by <, >} 
the Lie algebra of N with inner product <, > corresponding to the left-invariant metric on N. 

Suppose N admits a left-invariant Ricci soliton metric; that is, there exists g G GL(9T) such that 
g* <■,■ >—< g^^-,g^^- > is a nilsoliton. Moreover, suppose that <,>} admits a symmetric deriva- 
tion D G Der(O^) (D is symmetric with respect to <, > ). 

Let *n = ©OTa denote the eigenspace decomposition of D. Then the element g G Gi(Dl) such that g* <, > 
is nilsoliton can actually be chosen from the subgroup GL{m\-^) x • • • x GL(^Xk)- 

Remark. This theorem was known in the special case that the symmetric derivation D is the unique 
derivation such that the rank 1 extension s =< D > 001 admits a left-invariant Einstein metric [Heb98[ 
Proposition 6.8]. In fact, there it is shown that a slightly smaller group can be used. However, our theorem 
is very useful in practice when it is not known which symmetric derivation should be used to uniquely extend 
to an Einstein solvmanifold. 

Corollary 3.3. Let {iV, <, >} be a two-step nilpotent Lie group with Lie algebra Dl. Denote the center o/OT 
by 3- Then N is an Einstein nilradical if and only if there exist g G Gi(3^) x GL^^) such that {N, g* <, >} 
is a nilsoliton; here 3^ C is taken relative to <, >. 

Proof of Corollary. This corollary follows immediately from the theorem as every two-step nilpotent Lie 
algebra admits a symmetric derivation defined by Ld on 3^ a-ud 2Ld on 3- D 

Proof of Theorem \3.SX Let N = N^^ for some /i G V. We will apply Corollary [23] for the particular represen- 
tation at hand. 

By hypothesis, our nilpotent Lie algebra is an Einstein nilradical and so Theorem 13.11 implies that the 
G = GL(n,R) orbit is distinguished. Consider the subspace W = {\ V \ DX{X,Y) = \{DX,Y) + 
X{X, DY) for X,Y Q OT}. This is a vector subspace which contains /i as Z? G -Der(D^^). 

Lemma. Let a, /3 be eigenvalues D with corresponding eigenspaces V^Vp, then for A G W, X{Va, Vg) C 
Va+p, the eigenspace corresponding to a-\- (3. 

The proof is immediate. 

Define H to be the group GL(D^Ai) x • • • x GL{V\i^). First observe that H is closed under the metric 
adjoint with respect to <, >, hence H is reductive. 

For A G W , we will show m(A) G f) = LH by means of Equation 13.21 Let Xa (resp. Xp) G 01 be in the 
Aq (resp. \p) eigenspace of D, Aq i= Xp. Choose an orthonormal basis {Xi} consisting of eigenvectors of D, 
and apply Equation 13.21 This gives 

< m{X)X^,Xp > = -4^ < A(X„, Xi), X, >< X{Xp, X,), X, > 

-f2^ < X{X,,X,),X^ X X{X„Xj),Xp > 
= 

To see that this is zero, we observe that each summand is zero. In the first summation, we have < 
fi{Xa, Xi), Xj >< fi{Xp,Xi),Xj >— since either Aq -I- Ai ^ Xj or A^g -I- A^ ^ Xj as A^ ^ Xp; here we 
are applying the lemma above. Similarly, all the terms in the second summation are zero. Thus to(A) G I) 
for A G W^. 

Applying CoroUarv 12.41 together with Theorem 13. II completes the proof. □ 
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Theorem 3.4. Consider {N^ <, >} and a collection of symmetric derivations {Da} of^ (relative to <,>). 
Denote by the group which preserves the eigenspace decomposition of Da, as in Theorem \ 3.Sl Define 
H ~ nH" . The group H is a reductive algebraic group and N admits a nilsoliton metric if and only if 
{N, <, >} is such a metric for some h G H . 

Remark. It is not always true that the intersection of reductive groups is reductive, although such 
intersections are 'generically' reductive. See, e.g., [Jab08b| for an explicit example of this phenomenon. 

Proof. It was shown in the proof of the previous theorem that each H°' is closed under the metric adjoint. 
Hence, the same is true for the intersection H . Moreover, H is a, variety as it is the intersection of varieties. 
Thus, 77 is a reductive algebraic group. Let f) be the Lie algebra of H, then f) = nt)", where t)" is the Lie 
algebra of 

Define a subspace Wa ^ {\ e W \ DaX{X,Y) = X{DaX,Y) + \{X,DaY) for X,Y e m}. Define 
W = nWc(. The proof of the previous theorem shows that for X E W C Wa we have m(A) £ [)". Hence for 
XeW, m(A) e (). 

As before, applying Corollary [2T4] together with Theorem 13.11 completes the proof. □ 

Theorem 3.5. Suppose that ^ is a nilpotent algebra that can be written as a sum of ideals — Uli © D^2- 
Then N is an Einstein nilradical if and only if both D^i , OI2 o,re Einstein nilradicals. 

Remark. The property of being an Einstein nilradical is a property of the Lie group (or algebra) and 
does not depend on the choice of inner product on Dt. 

Proof. We choose to endow with an inner product so that Dli and ^2 are orthogonal. With this choice 
of inner product we have 01 = = ^H^i © ^H^^ where ^ = /ii + Ai2 e := (A^O^^ ® ^Hi) © (A^Ol^ ® 912) C 
A^DT* ® 91. As in the previous theorems, we will apply Corollarv l2. 41 together with Theorem 13. II 

We will show for X £W that m(A) G 0[(91i) x Qi{^2)- We apply Equation l3.2l bv choosing an orthonormal 
basis which respects 91i © 9^2 ■ For AT G 91i , F G 9l2 we have 

< m(A)A:,r >= -4^ < X{X,Xi),Xj >< X{Y,X,),Xj > +2^ < A(A:,, X^), AT >< X{X„Xj),Y> 

ij ij 

In the first summand, if Xi G 9Ti, then A(Y, X^) = 0; if A'^ G 9^2, then A(X, A'^) = 0. In the second summand, 
if A'i, Xj are not in the same subspace 9Ti or 9T2, then A(Ari, Xj) = 0. Now suppose, without loss of generality, 
that Xi^Xj G 9Ti. Then we have X{Xi,Xj) G 9Ti with is orthogonal to 9^2, hence < X{Xi^Xj)^Y >— 0. In 
this way, we see that < m{X)X, Y >= for AT G 91i, Y G 9^2- Thus m(A) G 0[(9Ti) x g[(9^2) and the proof 
is complete. □ 

Proposition 3.6. Consider 9^ = 9^i ©9^2- Let Vi denote the variety of Lie brackets on 9^i (see the remarks 
at the beginning of this section for more information on the set of Lie brackets). For fii € Vi, consider the 
set of Lie brackets on 9^ that can be written in the form 11 — fii (B fJ-2 ■ The moduli of such is precisely the 
moduli ofGL{mi) x GL(9^2) orbits in Vi x V2 C V. 

Proof. This follows immediately from the proof of the previous theorem and Theorem 12.11 see also the 
remarks following Theorem 12. II □ 

Remark. In this way we can construct moduli of Einstein and non- Einstein nilradicals. For example, pick 
one non- Einstein nilradical 91i (which are known to exist, see |LW07j or |Jab08cj ) and add on the vector 
space 9^2 • Letting the bracket vary on 9^2 will produce moduli of algebras 9Ti © 9T2 which cannot be Einstein 
nilradical by the previous proposition. However, the moduli of non-Einstein nilradicals constructed in this 
way is somewhat trivial. In [Jab08c| , there are given constructions of non- Einstein nilradicals which do not 
arise as direct sums, that is, the examples given are indecomposable. 

These are not the first examples of moduli of non-Einstein nilradicals. In |Wil08| . C. Will produces a 
curve of (pairwise) non-isomorphic, non-Einstein nilradicals. To our knowledge, these are the only other 
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known examples of moduli of this phenomenon. 

In the study of two-step nilpotent Lie algebras there are different representations that are very useful for 
obtaining concrete results. We will not go into the details of setting these up and refer the reader to [Jab08c| 
for the applications of this work in the two-step case. The work of this paper allows one to concretely 
construct moduli of Einstein and non-Einstein nilradicals in the set of "non-generic" two-step nilalgebras. 

Although we do not present the details here, one interesting example that comes from the two-step setting 
is the following. See the two-step case of type (2, 2k + 1) in |Jab08c| . 

Example 3.7. There exist representations with one generic orbit, but with moduli (of dimension > 1) of 
non-generic orbits in the null-cone. 

4 Nilpotent Orbits in the Adjoint Representation 

In this section we present the case of the adjoint representation. We focus our attention on the null-cone of 
this representation. Recall that the null-cone consists of a representation V consists of points v ^ V such 
that G G ■ V. When V = g is the adjoint representation, X S g is in the null-cone if and only if X is 
nilpotent, see, e.g., Remark 10.2 of [BHC62]. 

It is well-known that there exist only finitely many orbits in the null-cone of an adjoint representation of 
a reductive group. We reprove this result by applying Theorem 12.11 Moreover, we show that each of these 
orbits is distinguished (in the sense of Definition II. 6p . 

The known general proofs of the finiteness result are essentially the same as ours, without the language 
of the moment map. These proofs use Lemma 12.71 and an additional theorem on the reducibility of repre- 
sentations of reductive groups. These ideas originally go back to Richardson [Ric67| . 

Proposition 4.1. Let G = SLn{C). The null-cone of the adjoint action of G on g has finitely many orbits. 

These orbits are in one-to-one correspondence with the partitions of n; they correspond to the different 
Jordan normal forms with O's along the diagonal (the nilpotent cndomorphisms). See [Jan041 Section 1.1] 
for more details. 

Theorem 4.2 (Dynkin-Kostant). Let G be a (real or complex algebraic) linear reductive group. There exist 
only finitely many orbits in the null-cone of the adjoint representation. 

We prove this theorem in the complex setting and apply Theorem l2 . 1 21 to immediately obtain the theorem 
for reals groups once known for complex groups (cf. the end of Section [5]). 

Proof. It suffices to consider connected reductive groups G. As G is a linear group, G C SLn{C) for some 
n € N. Let 6* be a G-stable Cartan involution of S'L„(C), such exists by Proposition II. 21 Then g C sln(C) 
and we will endow sI„(C) with the inner product from Example 11.31 That is, < ■, ■ >= —B(-,6{-)) where 
B is the Killing form of s[„(C). This inner product is SU{n), K, and i-invariant; and so, this inner product 
satisfies the requirements of Section [1] We endow s[„(C) ~ L SLn{C) and g ~ LG with the same inner 
product inner product <<,>>=<,>. 
From this one readily computes 

m{X)^-[X,9{X)] (4.1) 

for X G 5l„(C). Observe that the SLn{C) orbits are G-detected along g (see Definition [L9]) ; that is, the 
moment map takes values in g when evaluated along g. 

We also observe that the G-nuU-cone in g is contained in the S'L„(C)-null-conc in s[„(C). Applying 
Theorem 12.11 we see that each S'L„(C) orbit through the G- null-cone consists of finitely many G-orbits. 
However, there are only finitely many such 5i„(C)-orbits (see Proposition 14. ip . Hence, there are only 
finitely many G-orbits in the G-nuU-cone. □ 

Theorem 4.3. Let G be a semi-simple group acting on g by the adjoint representation. Every orbit in the 
null-cone of g is distinguished (in the sense of Definition \L6\) . 
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Remark. This result has previously appeared in the literature, see Lemma 2.11 of |SV99| . The proof 
presented in that work uses the work of ISek87' . Our proof relies on constructing such critical points for the 
case of sln{C) and then applying Corollary [ 



Proposition 4.4. Every nilpotent orbit in the adjoint representation of SLn is distinguished. 
Proof. We first prove that the principal orbit in s[„ is distinguished. Consider the element 



Jn. — 



Ai 




A, 



A„_i 


We point out that this element is in the same SLn-orhit as the standard nilpotent Jordan block as long as 
the Xi are non-zero. 

By EquationO m{X) = -[X, -X*] = XX* - X*X and 

m{X) ■ X = [m{X), X] = m{X)X - Xm{X) = 2XX*X - X*X^ - X^X* 



We claim that A^ — \J ^(n — i) satisfies the condition m{Jn) • Jn = Jn (cf. Definition 11.61 and Lemma 
thereafter with c = 1) and hence SLn ■ Jn will be distinguished. This is easy to verify by direct computation. 

To see that every nilpotent orbit contains a distinguished point, observe that every nilpotent endomor- 
phism can be conjugated to be of the form 



X 



Jki 



Jkn 



where '^ki = n. This is the Jordan form where the super diagonal has non-zero entries which are rescaled. 
The matrix m{X) will have the same block decomposition and hence 



m{X)-X 



m{Jki) ■ Jk^ 



m{Jk2) ■ Jk2 



m{Jkj) ■ Jk, 



Jki 



Jk2 



X 



□ 



We continue now with the proof of the theorem for all semi-simple groups. We use the following obser- 
vation without proof. 



Lemma 4.5. Let X G g be in the null-cone. Then cX G G ■ X for all c G IK, where 
if Q is a real, respectively complex, Lie algebra. 



I, respectively C, 



First, we reduce to the case that G is simple. Suppose that the result is known for simple groups and 
decompose g = gi x . . . g^. as a product of simple factors. Let X be a nilpotent element of g, then X = J2^i 
where Xi e g^ are all nilpotent elements. Observe that m{X) = ^ m{Xi). Using the lemma above and the 
hypothesis, we may assume that each Xi satisfies m{Xi) ■ Xi = Xi. But now m{X) ■ X = ^TO(Xi) ■ Xi = 

EX^=X. 

All that remains to be shown is that the theorem is true in the case G is simple. Firstly, we embed G 
into SLn so that G is self-adjoint, as in the beginning of the section. As G is simple, any i^-invariant inner 
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product on p is unique up to scaling. This is due to the fact that K acts irreducibly on p. Thus, given 
any inner product on g, as in Section [Tl our moment map is uniquely defined up to rescaling. As rescaling 
the moment map does not affect the property of a point being distinguished, we may assume that our inner 
product on g is the restriction of the inner product on s[„. 

Recall that the S'Ln-orbits are G-detected along g. As every nilpotent element of g is a nilpotent element 
of sin and every nilpotent orbit of SLn is S'Ln-distinguished, applying Corollary 12.41 we see that every 
nilpotent orbit of G is G-distinguished. 
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